The present invention relates to a multimedia system and software for creating, storing, organizing, integrating, and displaying recorded or live visual effects with recorded or live music performances.
Attempts to enhance music performances, either live or recorded, by adding visual effects or visual interpretations of music have been made using a number of diferent methods and visual media. For example, elaborate lighting systems, manual or computer controlled, used to create moods and punctuate the rhythm and dynamics of live performances are known in the art. In another example, displaying still pictures or moving pictures on large screens as a backdrop to live performances is also known in the art. Furthermore, conventional multimedia systems that combine music with images, text, animation, video, and other similar effects have been used for recorded or live performances.
Recently, as a natural progression of the above examples, there has been interest in integrating computer generated graphics as visual interpretations of music. Computer generated graphics may include abstract, flowing graphic images created in realtime by software running on a computer, still photographs in digital format, three-dimensional computer graphical objects and/or moving-video. With respect to the moving-video visual interpretations, there have been attempts to allow for some level of rhythmic interpretation of the music by using switchers to xe2x80x9ccutxe2x80x9d in traditional editing style, e.g., inserting one non-synchronized video source to another. These methods, however, involve pre-established algorithms to integrate the video effect and music, and do not provide for xe2x80x9crealtimexe2x80x9d editing or integrating of the video effect to synchronize with the music.
The above described conventional systems, therefore, have not furnished a satisfactory solution to provide a visual performance artist (i.e., a person adding the visual interpretation to music performances) with tools necessary to allow substantially interactive and interpretive control over the visual media, while maintaining the ability to maintain synchronization of music. Most often, the visual interpretations are prerecorded and cannot be altered during recorded or live music performances.
The use of MIDI (Musical Instrument Digital Interface), a standard for controlling electronic musical instruments, has been a logical choice as a means to control the visual interpretations in the context of live performances. Although using MIDI addresses the means of interfacing the computers and performers involved, it does not solve technical problems of creating visual software that can respond quickly or xe2x80x9con-the-flyxe2x80x9d /realtime to interpret the MIDI control commands that occur at a very rapid pace when rhythm-based music is interpreted by a visual performance artist.
This has been especially problematic when attempting to use digitized moving-video. Current tools are incapable of providing full screen video being triggered in realtime without a response latency which visual performers find unacceptable. In other words, a performer cannot, with the current tools, select and trigger a full-screen video clip, and have it appear immediately on the screen without latency. This makes it difficult, if not impossible, to xe2x80x9cplayxe2x80x9d video clips interpretively, in synchronization or rhythm with the music. The result is a visual performance which seems xe2x80x9cdetachedxe2x80x9d from the music. This results in less opportunity to enhance and complement the musical performance. In fact, the visual elements may become distracting.
In the above-described conventional systems, the desired individual visual images are required to be loaded to Random Access Memory (RAM) in the computer, and therefore, the loading and unloading of individual images and the music results in an unacceptable latency or delay. Moreover, these systems are constrained in the amount of visual images that can be created, stored, accessed, and displayed.
The present invention provides a multimedia production and performance system or xe2x80x9csoftware toolxe2x80x9d comprising software that allows the performing visual artist to create a palette of one set or multiple sets of digitized visual images, select a visual image or images from a palette, and have those images displayed substantially instantaneously on the performance surface, such that the performer may xe2x80x9cplayxe2x80x9d the visual images rhythmically and in synchronization with the music performance like an instrument.
The present invention also allows the visual artist to use either a standard computer keyboard or MIDI keyboard to interact with the software tool, such that striking the keys allows the artist to access and display a visual image by playing the keyboard like an instrument.
In addition, the present invention allows the visual artist to record his/her visual image performances into a standard MIDI sequencer such that the performances may be saved and recorded on CD-ROM or Digital Versatile Disc (DVD), and edited as part of the creative musical process, without the need for or knowledge of traditional digital video editing equipment and software. The performances can then be played back at a later time on a standard CD or DVD player, in synchronization with other pre-recorded musical or visual elements.
In a live concert or club environment, the system transforms the music performance into a multidimensional, multi-sensory experience, with surround audio and multiple surfaces of synchronized visuals enveloping the visitor. In the studio, the system functions to create xe2x80x9cvideo re-mixesxe2x80x9d for DVD, enhancing pre-existing audio and video with surround mixes and multiple, selectable synchronized visual tracks.
In particular, the present invention includes a method for integrating music performance with visual effects. The method includes the step of storing a plurality of items each of which contains a visual image, allowing a first user to select at least one subset of the plurality of items and forming each of the at least one selected subset of the plurality of items into a file. The method also includes the steps of allowing a second user to select one of the files, allowing the second user to select one item in the selected file, and displaying or xe2x80x9cplayingxe2x80x9d the selected item as the music performance is being played.